Headtracked processing for headtracked playback of audio signals

ABSTRACT

A method of simulating a spatial sound environment to a listener over headphones is disclosed comprising inputting a series of sound signals having spatial components; determining a current orientation of the headphones around the listener; determining a mapping function from a series of spatially static virtual speakers placed around the listener to each ear of the listener; utilising the current orientation to determine a current panning of the sound signals to the series of virtual speakers so as to produce a panned sound input signal for each of the virtual speakers; utilising the mapping function to map the panned sound input signal to each ear of the listener, and combining the mapped panned sound input signals to produce a left and right output signal for the headphones.

FIELD OF THE INVENTION

The present invention relates to the creation of spatialized soundsutilizing a headtracked set of headphones.

BACKGROUND OF THE INVENTION

Methods for localizing sounds utilizing headphones and a headtrackingunit are known. For example, in U.S. patent Ser. No. 08/723,614 entitled“Methods and Apparatus for Processing Spatialized Audio”, there isdisclosed a system for virtual localization of a sound field around alistener utilizing a pair of headphones and a headtracking unit whichdetermines the orientation of the headphones relative to an externalenvironment. Unfortunately, the disclosed arrangement requires a highcomputational power or resource for real time rotation of a sound fieldenvironment so as to take into account any headphone movement relativeto the desired sound field output.

Alternatively, without headtracking, a virtual speaker system overheadphones can be simulated by using a pair of filters for each virtualsound source and then a post mixing of the results to produce left andright signals. For example, turning initially to FIG. 1, if it isdesired to simulate to a user 1 over headphones eg. 2, 3 a virtual soundenvironment, with, for example, the environment comprising the popularDolby DIGITAL (Trade Mark) environment which includes a left, 5, andright, 6 sound source in addition to a center cell source 7 and backleft and right sound sources 8 and 9, then one form of suitablearrangement may be as illustrated 10 in FIG. 2. The arrangement 10includes, for each channel eg. 11 providing a head related transferfunction filter eg. 12, 13 for each input channel which maps the soundsource to each of the left and right ears so as to form left and rightheadphone channels 16, 17. Similarly, each of the other channels issimilarly processed and the output summed to each head channel. Thearrangement 10 in FIG. 2 is provided for a system that does not utilizeheadtracking. The arrangement of FIG. 2 requires significant lengthfilters eg. 12, 13 for each channel. Of course, many filteroptimisations are possible in respect of the non treadtrackedarrangement. An example of these optimisations include those disclosedin PCT Patent Application No. PCT AU99/00002 filed 6 Jan., 1999 by thepresent applicant entitled “Audio Signal Processing Method andApparatus”.

One possible method utilized by others to perform headtracking is to usean enormous amount of computational memory for storing a large number ofsets of filter coefficients. For example, a set of filter coefficientscould be stored for every angle around a listener (for full 360coverage), then, each time the listener rotated their head the filtercoefficients could be updated to reflect the new angle. A cross fade tothe new filter coefficients would remove any unwanted artefacts. Thistechnique has the significant disadvantage that it requires an enormousamount of memory to store the large number of filtered coefficients.

An alternative technique is disclosed in U.S. Pat. No. 5,659,619 by Abelwhich utilizes a process of principle component analysis where the headrelated transfer function is assumed to consist of several individualfilter structures which are all modified from a look-up table accordingto a current head angle. This method provides for a reduction in memoryrequirements.

However, it is only practical for short filters (short HRTF length)which provide for directionality of a sound source and it is notpractical for a full room reverberant response in addition to theeffective simulation of a full room.

It would be desirable to provide for a more efficient form of simulationof a sound surround environment over headtracked headphones in additionto the effective simulation of a full room reverberant response.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide for a more efficientform of simulation of a surround sound environment over headtrackedheadphones.

In accordance with a first aspect of the present invention, there isprovided a method of simulating a spatial sound environment to alistener over headphones comprising inputting a series of sound signalshaving spatial components; determining a current orientation of theheadphones around the listener; determining a mapping function from aseries of spatially static virtual speakers placed around the listenerto each ear of the listener; utilising the current orientation todetermine a current panning of the sound signals to the series ofvirtual speakers so as to produce a panned sound input signal for eachof the virtual speakers; utilising the mapping function to map thepanned sound input signal to each ear of the listener; and combining themapped panned sound input signals to produce a left and right outputsignal for the headphones.

Preferably, the virtual speakers include a set of simulated speakersplaced at substantially equal angles around the listener which can beplaced substantially in a horizontal plane around a listener or placedso as to fully surround a listener in three dimensions. The presentinvention has particular application wherein the series of sound signalscomprise a Dolby DIGITAL encoding of a sound environment.

In accordance with a second aspect of the present invention, there isprovided an apparatus for simulating a spatial sound environment to alistener over headphones comprising input means for inputting a seriesof signals comprising a spatial sound environment; panning means forpanning the series of signals amongst a predetermined number of virtualoutput signals to produce a plurality of virtual output speakerssignals; head related transfer function mapping means for mapping thevirtual output speaker signals to left and right headphone channelsignals; and combining means for combining each of the left and rightheadphone channel signals into combined left and right headphone signalsfor playback over the headphones.

Preferably, the panning means, the head related transfer functionmapping means and the combining means are implemented in the form of asuitably programmed digital signal processor.

BRIEF DESCRIPTION OF THE DRAWINGS

Notwithstanding any other forms which may fall within the scope of thepresent invention, preferred forms of the invention will now bedescribed, by way of example only, with reference to the accompanyingdrawings in which:

FIG. 1 illustrates the concept of a surround sound system;

FIG. 2 illustrates a prior art arrangement for creating a surround soundenvironment over headphones;

FIG. 3 illustrates the utilization of a virtual speaker system inaccordance with the preferred embodiment;

FIG. 4 is a schematic block diagram of the structure of the preferredembodiment;

FIGS. 5 and 6 illustrate the extension of the preferred embodiment tothree dimensions; and

FIG. 7 illustrates one form of implementation of the preferredembodiment.

DESCRIPTION OF PREFERRED AND OTHER EMBODIMENTS

In the preferred embodiment, a fixed filter and coefficient structure isutilized to simulate a stationary virtual speaker array and then aspeaker panner is utilized to position the virtual sound sources atdesired positions. The preferred embodiment will be discussed withreference to a Surround Sound implementation of the popular DolbyDIGITAL format.

Turning to FIG. 3, there is illustrated a method of the preferredembodiment. The method of the preferred. embodiment comprises utilizinga set of virtual speakers 21-26 arranged around a listener 27. A headrelated transfer function to each ear of the listener 27 is calculatedfor each of the virtual speakers 21-26 arranged around a listener 27.The techniques utilized can be substantially the same as those describedpreviously with reference to FIG. 2 and known in the prior art.

A series of virtual surround sound speakers 31-35 are then utilizedhaving a stable external reference frame relative to the user 27. Hence,as the user 27 turns their head, the virtual speaker 32 for example ispanned between speakers 21-22 so as to locate the speaker 32 at therequisite point between speakers 21 and 22. Similar panning occurs foreach of the other virtual surround sound speakers 32-35. Hence, each ofthe surround sound channel sources eg. 32 is panned between speakers soas to provide for the directionality of each sound source. Thedirectionality of each sound source can be updated depending on therotation of a listener's head and the speaker panning technique can betotally flexible and compatible with prior art panning techniques forconventional loudspeakers.

Turning now to FIG. 4, there is illustrated one form of arrangement ofthe preferred embodiment 40. The preferred embodiment is based aroundtwo parts including a speaker panning section 41 and HRTF section 42.The HRTF section 42 includes the usual series of filters eg. 43, 44which map each of the virtual speakers 21-26 to the left and right earof the listener 27. The filter coefficients being substantially static.

The input channels for each of the surround sound sources 31-35 areinput to an N input to M output speaker panner 46. The speaker panner 46also having as an input 47 the headtracking input signal from alistener's headphone. The speaker panner 46 can then be set to providepanning between the virtual output speakers 21-26 which are output eg.49.

The technique of the preferred embodiment can be extended to provide forheadtracking of elevation and roll of a user's head position where suchinformation is available from the headtracking unit. This can beachieved by extending the location of the stationary virtual speakers tobe in a three-dimensional cube around a listener. For example, if eightvirtual speakers are simulated representing the eight corners of a cubearound a listener, then any panning system can also compensate for headmovements around a Y and Z plane. Hence, in addition to yaw, elevationand roll can also be taken into account. Of course, the more virtualspeakers utilized to create the virtual speaker space around a listener,the better the accuracy of the system. Once again, panning can beprovided by means of a front end system that utilizes the headtrackedyaw, elevation and roll position to determine the panning effect betweenspeakers. For example, as illustrated in FIG. 5, the elevation of alistener 55 can be determined via a standard headtracking unit andutilized to pan three-dimensional sound sources 56-59 around speakers50-53 in accordance with the requirements. Similarly, as illustrated inFIG. 6, the roll of a user's head 55 can be utilized for panning thevirtual sound sources 66-69 between virtual speakers 61-64 again as apre-processing step.

Turning now to FIG. 7, there is illustrated an example system 70 forimplementation of the preferred embodiment. The system 70 includes astandard DVD digital input source 71 which is fed to an DIGITAL decoder72 which again can be standard. The DIGITAL decoder outputs centerchannel 73, front left and right channels 74, and surround or back leftand right channels 75. The outputs 73-75 are fed to a DSP processingboard 76 which operates with an attached memory 77. One form of suitableDSP processing board can be the Motorola 56002 EVM evaluation board carddesigned to be inserted into a PC type computer and directly programmedtherefrom and having suitable Analogue/Digital and Digital/Analogueconverters.

A set of headphones 79 are provided which include headtrackingcapabilities in the form of an angular position circuit 80. The angularposition circuit 80 determines the yaw, elevation and roll and cancomprise a Polhemus 3 space Insidetrak Tracking system available fromPolhemus, 1 Hercules Drive, PO Box 560, Colchester, Vt. 05446, USA. Theoutput from the angular position circuit 80 is converted to a digitalform 81 for inputting to DSP chip 76. The DSP chip 76 is responsible forimplementing the core functionality of FIG. 4, outputting two digitalchannels to digital to analogue converter 82 which in turn outputsanalogue left and the right headphone speaker channel data which can beamplified 83, 84 in accordance with the requirements. The DSP chip 76also implements the speaker panner mixing which pans the input sources73-75 according to the input angular position. Further, a filter arrayis provided within the DSP 76 which simulates the virtual speaker arrayof six speakers in accordance with the previously known prior arttechniques.

It would be therefore evident that the preferred embodiment provides fora simplified form of providing for full surround sound capabilities ofthe headtracked headphones in the presence of movement of the listener'shead.

It would be appreciated by a person skilled in the art that numerousvariations and/or modifications may be made to the present invention asshown in the specific embodiment without departing from the spirit orscope of the invention as broadly described. The present embodiment is,therefore, to be considered in all respects to be illustrative and notrestrictive.

I claim:
 1. A method of simulating a spatial sound environment to alistener over headphones comprising: inputting a series of sound signalshaving spatial components; determining a current orientation of saidheadphones around said listener; determining a mapping function from aseries of spatially static virtual speakers placed around the listenerto each ear of the listener; utilising said current orientation todetermine a current panning of said sound signals to said series ofvirtual speakers so as to produce a panned sound input signal for eachof said virtual speakers; utilising said mapping function to map saidpanned sound input signal to each ear of said listener; and combiningsaid mapped panned sound input signals to produce a left and rightoutput signal for said headphones.
 2. A method as claimed in claim 1wherein said virtual speakers include a set of simulated speakers placedat substantially equal angles around said listener.
 3. A method asclaimed in claim 1 wherein said virtual speakers are substantially in ahorizontal plane around a listener.
 4. A method as claimed in claim 1wherein said virtual speakers are placed so as to fully surround alistener in three dimensions.
 5. A method as claimed in claim 1 whereinsaid series of sound signals comprise a Dolby DIGITAL encoding of asound environment.
 6. An apparatus for simulating a spatial soundenvironment to a listener over headphones comprising: input means forinputting a series of signals comprising a spatial sound environment forlistening in a first reference frame; panning means for panning saidseries of signals amongst a predetermined number of virtual outputsignals to produce a plurality of panned virtual output speakers signalsin a second reference frame that is fixed relative to the orientation ofsaid headphones, said panning means accepting a signal indicative of theorientation of said headphones to said first reference fame; headrelated transfer function mapping means for mapping said panned virtualoutput speaker signals to left and right headphone channel signals; andcombining means for combining each of said left and right headphonechannel signals into combined left and right headphone signals forplayback over said headphones, such that the head related transferfunction mapping means and the means for combining need not vary fordifferent orientations of said headphones to said first reference frame.7. An apparatus as claimed in claim 6 wherein said panning means, saidhead related transfer function mapping means and said combining meansare implemented in the form of a suitably programmed digital signalprocessor.
 8. An apparatus for simulating a spatial sound environment toa listener over headphones comprising: an input device adapted to inputa series of signals comprising a spatial sound environment for listeningin a first reference frame; a panning module adapted to pan said seriesof signals amongst a predetermined number of virtual output signals toproduce a plurality of panned virtual output speakers signals in asecond reference frame that is fixed relative to the orientation of saidheadphones, said panning module accenting a signal indicative of theorientation of said headphones to said first reference frame; a headrelated transfer output mapping module adapted to map said pannedvirtual output speaker signals to left and right headphone channelsignals; and a combining module adapted to combine each of said left andright headphone channel signals into combined left and right headphonesignals for playback over said headphones, such that the head relatedtransfer function mapping module and the combining module need not varyfor different orientations of said headphones to said first referencefame.
 9. An apparatus as claimed in claim 8, wherein said panningmodule, said head related transfer function mapping module and saidcombining module are implemented in the form of a suitable programmeddigital signal processor.
 10. An apparatus as claimed in claim 8,wherein said virtual output speaker signals correspond to virtualspeakers which include a set of simulated speakers placed atsubstantially equal angles around said listener.
 11. An apparatusclaimed in claim 10, wherein said virtual speakers are substantially ina horizontal plane around a listener.
 12. An apparatus as claimed inclaim 10, wherein said virtual speakers are placed so as to fullysurround a listener in three dimensions.
 13. A method as claimed inclaim 8, wherein said series of signals comprise a Dolby DIGITALencoding of a sound environment.